{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 拆分学习—Bank Marketing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在这个教程中，我们将以银行的市场营销模型为例，展示在`SecretFlow`框架下如何完成垂直场景下的拆分学习。\n",
    "`SecretFlow`框架提供了一套用户友好的Api，可以很方便的将您的keras模型或者pytorch模型应用到拆分学习场景，成为拆分学习模型，进而完成垂直场景的联合建模任务。\n",
    "在接下来的教程中我们将手把手演示，如何将您已有的`keras`模型变成`secretflow`下的拆分学习模型，完成联邦多方建模任务。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 什么是拆分学习？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "拆分学习的核心思想是将网络结构进行拆分，每个设备（机构）只保留一部分网络结构，所有设备的子网络结构组合在一起，构成一个完整的网络模型。在训练过程中，不同的设备（机构）只对本地的网络结构进行前向或反向计算，并将计算结果传递给下一个设备，多个设备端通过联合模型，完成训练，直到收敛为止。\n",
    " <img alt=\"split_learning_tutorial.png\" src=\"resource/split_learning_tutorial.png\" width=\"600\">  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Alice`：拥有`data_alice`，`model_base_alice`  \n",
    "`Bob`: 用于`data_bob`，`model_base_bob`，`model_fuse`  \n",
    "\n",
    "1. `Alice`方用本方的数据经过`model_base_alice`得到`hidden0`，发送给Bob\n",
    "2. `Bob`方用本方的数据经过`model_base_bob`得到`hidden1`\n",
    "3. `hidden0`和`hidden1`输入给`Agg Layer`层做聚合，输出聚合后的`hidden_merge`\n",
    "4. `Bob`方将`hidden_merge`输入给`model_fuse`结合`label`得到梯度，并进行回传\n",
    "5. 梯度经过`AggLayer`拆分成两部分`g0`,`g1`，将`g0`和`g1`分别发送给`Alice`和`Bob`\n",
    "6. `Alice`和`Bob`的`basenet`分别根据`g0`和`g1`对本方的模型进行更新  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 任务介绍"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "市场营销是银行业在不断变化的市场环境中，为满足客户需要、实现经营目标的整体性经营和销售的活动。在目前大数据的环境下，数据分析为银行业提供了更有效的分析手段。在客户需求分析，了解目标市场趋势以及更宏观的市场策略都能提供依据与方向。  \n",
    "  \n",
    "此数据来源于[kaggle](https://www.kaggle.com/janiobachmann/bank-marketing-dataset)是一组经典的银行市场营销数据，是葡萄牙一家银行机构电话直销活动，目标变量为客户是否订阅定期存款。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据介绍\n",
    "\n",
    "1. 样本量总计11162个，其中训练集8929， 测试集2233\n",
    "2. 特征16维，标签维2分类\n",
    "3. 我们预先对数据进行了切割，alice持有其中的4维基础属性特征，bob持有12维银行交易特征，对应的label只有alice方持有"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们先来看看我们银行市场营销数据长什么样的？  \n",
    "\n",
    "原始数据经过分拆后分成bank_alice和bank_bob，分别存在alice和bob两方。这里的csv是原始数据仅经过分拆，没有做预处理的数据  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "import io\n",
    "import pandas as pd\n",
    "data_dict = {'alice': 'bank.csv',\n",
    "               'bob': 'bank.csv'}\n",
    "dataset_dict = {}\n",
    "for device, url in data_dict.items():\n",
    "    response = requests.get(url)\n",
    "    response.raise_for_status()\n",
    "    dataset_dict[device] = pd.read_csv(io.BytesIO(response.content))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们假设Alice是一个新银行，他们只有用户的基本信息，和买来的是否购买过理财产品的label"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>age</th>\n",
       "      <th>job</th>\n",
       "      <th>marital</th>\n",
       "      <th>education</th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>59</td>\n",
       "      <td>admin.</td>\n",
       "      <td>married</td>\n",
       "      <td>secondary</td>\n",
       "      <td>yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>56</td>\n",
       "      <td>admin.</td>\n",
       "      <td>married</td>\n",
       "      <td>secondary</td>\n",
       "      <td>yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>41</td>\n",
       "      <td>technician</td>\n",
       "      <td>married</td>\n",
       "      <td>secondary</td>\n",
       "      <td>yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>55</td>\n",
       "      <td>services</td>\n",
       "      <td>married</td>\n",
       "      <td>secondary</td>\n",
       "      <td>yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>54</td>\n",
       "      <td>admin.</td>\n",
       "      <td>married</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11157</th>\n",
       "      <td>11157</td>\n",
       "      <td>33</td>\n",
       "      <td>blue-collar</td>\n",
       "      <td>single</td>\n",
       "      <td>primary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11158</th>\n",
       "      <td>11158</td>\n",
       "      <td>39</td>\n",
       "      <td>services</td>\n",
       "      <td>married</td>\n",
       "      <td>secondary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11159</th>\n",
       "      <td>11159</td>\n",
       "      <td>32</td>\n",
       "      <td>technician</td>\n",
       "      <td>single</td>\n",
       "      <td>secondary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11160</th>\n",
       "      <td>11160</td>\n",
       "      <td>43</td>\n",
       "      <td>technician</td>\n",
       "      <td>married</td>\n",
       "      <td>secondary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11161</th>\n",
       "      <td>11161</td>\n",
       "      <td>34</td>\n",
       "      <td>technician</td>\n",
       "      <td>married</td>\n",
       "      <td>secondary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>11162 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          id  age          job  marital  education    y\n",
       "0          0   59       admin.  married  secondary  yes\n",
       "1          1   56       admin.  married  secondary  yes\n",
       "2          2   41   technician  married  secondary  yes\n",
       "3          3   55     services  married  secondary  yes\n",
       "4          4   54       admin.  married   tertiary  yes\n",
       "...      ...  ...          ...      ...        ...  ...\n",
       "11157  11157   33  blue-collar   single    primary   no\n",
       "11158  11158   39     services  married  secondary   no\n",
       "11159  11159   32   technician   single  secondary   no\n",
       "11160  11160   43   technician  married  secondary   no\n",
       "11161  11161   34   technician  married  secondary   no\n",
       "\n",
       "[11162 rows x 6 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset_dict['alice']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pandas.core.frame.DataFrame"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(dataset_dict['alice'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "bob端是一个老银行，他们有用户的账户余额，是否有房，是否有贷款，以及最近的营销反馈"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>default</th>\n",
       "      <th>balance</th>\n",
       "      <th>housing</th>\n",
       "      <th>loan</th>\n",
       "      <th>contact</th>\n",
       "      <th>day</th>\n",
       "      <th>month</th>\n",
       "      <th>duration</th>\n",
       "      <th>campaign</th>\n",
       "      <th>pdays</th>\n",
       "      <th>previous</th>\n",
       "      <th>poutcome</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>no</td>\n",
       "      <td>2343</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>unknown</td>\n",
       "      <td>5</td>\n",
       "      <td>may</td>\n",
       "      <td>1042</td>\n",
       "      <td>1</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>unknown</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>no</td>\n",
       "      <td>45</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>unknown</td>\n",
       "      <td>5</td>\n",
       "      <td>may</td>\n",
       "      <td>1467</td>\n",
       "      <td>1</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>unknown</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>no</td>\n",
       "      <td>1270</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>unknown</td>\n",
       "      <td>5</td>\n",
       "      <td>may</td>\n",
       "      <td>1389</td>\n",
       "      <td>1</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>unknown</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>no</td>\n",
       "      <td>2476</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>unknown</td>\n",
       "      <td>5</td>\n",
       "      <td>may</td>\n",
       "      <td>579</td>\n",
       "      <td>1</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>unknown</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>no</td>\n",
       "      <td>184</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>unknown</td>\n",
       "      <td>5</td>\n",
       "      <td>may</td>\n",
       "      <td>673</td>\n",
       "      <td>2</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>unknown</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11157</th>\n",
       "      <td>11157</td>\n",
       "      <td>no</td>\n",
       "      <td>1</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>cellular</td>\n",
       "      <td>20</td>\n",
       "      <td>apr</td>\n",
       "      <td>257</td>\n",
       "      <td>1</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>unknown</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11158</th>\n",
       "      <td>11158</td>\n",
       "      <td>no</td>\n",
       "      <td>733</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>unknown</td>\n",
       "      <td>16</td>\n",
       "      <td>jun</td>\n",
       "      <td>83</td>\n",
       "      <td>4</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>unknown</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11159</th>\n",
       "      <td>11159</td>\n",
       "      <td>no</td>\n",
       "      <td>29</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>cellular</td>\n",
       "      <td>19</td>\n",
       "      <td>aug</td>\n",
       "      <td>156</td>\n",
       "      <td>2</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>unknown</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11160</th>\n",
       "      <td>11160</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>no</td>\n",
       "      <td>yes</td>\n",
       "      <td>cellular</td>\n",
       "      <td>8</td>\n",
       "      <td>may</td>\n",
       "      <td>9</td>\n",
       "      <td>2</td>\n",
       "      <td>172</td>\n",
       "      <td>5</td>\n",
       "      <td>failure</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11161</th>\n",
       "      <td>11161</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>cellular</td>\n",
       "      <td>9</td>\n",
       "      <td>jul</td>\n",
       "      <td>628</td>\n",
       "      <td>1</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>unknown</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>11162 rows × 13 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          id default  balance housing loan   contact  day month  duration  \\\n",
       "0          0      no     2343     yes   no   unknown    5   may      1042   \n",
       "1          1      no       45      no   no   unknown    5   may      1467   \n",
       "2          2      no     1270     yes   no   unknown    5   may      1389   \n",
       "3          3      no     2476     yes   no   unknown    5   may       579   \n",
       "4          4      no      184      no   no   unknown    5   may       673   \n",
       "...      ...     ...      ...     ...  ...       ...  ...   ...       ...   \n",
       "11157  11157      no        1     yes   no  cellular   20   apr       257   \n",
       "11158  11158      no      733      no   no   unknown   16   jun        83   \n",
       "11159  11159      no       29      no   no  cellular   19   aug       156   \n",
       "11160  11160      no        0      no  yes  cellular    8   may         9   \n",
       "11161  11161      no        0      no   no  cellular    9   jul       628   \n",
       "\n",
       "       campaign  pdays  previous poutcome  \n",
       "0             1     -1         0  unknown  \n",
       "1             1     -1         0  unknown  \n",
       "2             1     -1         0  unknown  \n",
       "3             1     -1         0  unknown  \n",
       "4             2     -1         0  unknown  \n",
       "...         ...    ...       ...      ...  \n",
       "11157         1     -1         0  unknown  \n",
       "11158         4     -1         0  unknown  \n",
       "11159         2     -1         0  unknown  \n",
       "11160         2    172         5  failure  \n",
       "11161         1     -1         0  unknown  \n",
       "\n",
       "[11162 rows x 13 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset_dict['bob']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 环境的搭建"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在secretflow环境创造2个实体[Alice，Bob]  \n",
    "其中 `alice`, `bob` 是两个PYU  \n",
    "构造好两个对象后就可以愉快的开始拆分学习的玩耍了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "E0315 16:57:27.186333308   69424 fork_posix.cc:70]           Fork support is only compatible with the epoll1 and poll polling strategies\n",
      "E0315 16:57:27.202971759   69424 fork_posix.cc:70]           Fork support is only compatible with the epoll1 and poll polling strategies\n",
      "E0315 16:57:27.212423932   69424 fork_posix.cc:70]           Fork support is only compatible with the epoll1 and poll polling strategies\n"
     ]
    }
   ],
   "source": [
    "import secretflow as sf\n",
    "\n",
    "sf.init(['alice', 'bob'], num_cpus=8, log_to_driver=True)\n",
    "alice, bob = sf.PYU('alice'), sf.PYU('bob')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 导入训练所需要的库  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-03-15 16:58:34.813464: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib:/opt/rh/devtoolset-10/root/usr/lib64/dyninst:/opt/rh/devtoolset-10/root/usr/lib/dyninst:/opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib\n"
     ]
    }
   ],
   "source": [
    "from secretflow.data.split import train_test_split\n",
    "from secretflow.model.sl_model import SLModelTF"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 准备训练数据 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**构建联邦表**  \n",
    "联邦表是一个横跨多方的虚拟概念，我们针对垂直场景定义了`VDataFrame`  \n",
    "1. 联邦表的各方数据是存储在**各方本地，不允许出域**\n",
    "2. 除了拥有数据的那一方，其他人都**不会接触**的具体的数据存储\n",
    "3. 对于联邦表的任何操作，会由driver调度到各个worker去执行，执行指令会层层下发，直到具体worker的python runtime才会执行，框架保证了只有worker.device和object.device相同时能够操作数据。\n",
    "4. 联邦表的设计从中心视角提供了对多方数据的管理和操作\n",
    "5. 联邦表的接口对齐`pandas.DataFrame`,降低用户操作多方数据成本\n",
    "6. secretflow框架提供了明密文混合编程能力，垂直联邦表在构建时会使用PPU，利用MPC-PSI对各方数据进行安全求交并对齐\n",
    "\n",
    "<img alt=\"vdataframe.png\" src=\"resource/vdataframe.png\" width=\"600\">  \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`secretflow`的VDataFrame提供了类似pandas的readcsv接口，不同的是，read_csv接收的一个字典（定义双方数据的路径），我们可以使用`secretflow.vertical.read_csv`来构建垂直联邦表VDataFrame\n",
    "```\n",
    "read_csv(file_dict,delimiter,ppu,keys,drop_key)\n",
    "  其中介绍几个比较关键的参数\n",
    "    filepath: 参与方文件地址，地址可以是相对或绝对路径的本地文件\n",
    "    ppu:PPU设备，用于PSI数据对齐；若不指定，则默认数据预对齐\n",
    "    keys:用于对齐的列名，支持多关联键求交\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "创建ppu,用于后续的安全求交，对齐"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "ppu = sf.PPU(sf.utils.testing.cluster_def(['alice', 'bob']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[2m\u001b[36m(PPURuntime pid=73210)\u001b[0m I0315 16:58:41.919611 73210 external/com_github_brpc_brpc/src/brpc/server.cpp:1046] Server[ppu::link::internal::ReceiverServiceImpl] is serving on port=49793.\n",
      "\u001b[2m\u001b[36m(PPURuntime pid=73210)\u001b[0m I0315 16:58:41.919692 73210 external/com_github_brpc_brpc/src/brpc/server.cpp:1049] Check out http://i85c08157.eu95sqa:49793 in web browser.\n",
      "\u001b[2m\u001b[36m(PPURuntime pid=73212)\u001b[0m I0315 16:58:41.882383 73212 external/com_github_brpc_brpc/src/brpc/server.cpp:1046] Server[ppu::link::internal::ReceiverServiceImpl] is serving on port=13967.\n",
      "\u001b[2m\u001b[36m(PPURuntime pid=73212)\u001b[0m I0315 16:58:41.882458 73212 external/com_github_brpc_brpc/src/brpc/server.cpp:1049] Check out http://i85c08157.eu95sqa:13967 in web browser.\n",
      "\u001b[2m\u001b[36m(PPURuntime pid=73212)\u001b[0m I0315 16:58:41.983224 106636 external/com_github_brpc_brpc/src/brpc/socket.cpp:2202] Checking Socket{id=0 addr=127.0.0.1:49793} (0x56527cdd3600)\n",
      "\u001b[2m\u001b[36m(PPURuntime pid=73212)\u001b[0m I0315 16:58:41.983416 106636 external/com_github_brpc_brpc/src/brpc/socket.cpp:2262] Revived Socket{id=0 addr=127.0.0.1:49793} (0x56527cdd3600) (Connectable)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[2m\u001b[36m(PPURuntime pid=73212)\u001b[0m [2022-03-15 16:58:41.882] [info] [context.cc:58] connecting to mesh, id=root, self=1\n",
      "\u001b[2m\u001b[36m(PPURuntime pid=73212)\u001b[0m [2022-03-15 16:58:41.898] [info] [context.cc:83] try_connect to rank 0 not succeed, sleep_for 1000ms and retry.\n",
      "\u001b[2m\u001b[36m(PPURuntime pid=73210)\u001b[0m [2022-03-15 16:58:41.919] [info] [context.cc:58] connecting to mesh, id=root, self=0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[2m\u001b[36m(_run pid=73204)\u001b[0m 2022-03-15 16:58:42,649,649 DEBUG [connectionpool.py:_new_conn:1001] Starting new HTTPS connection (1): federal.oss-cn-hangzhou.aliyuncs.com:443\n",
      "\u001b[2m\u001b[36m(_run pid=73204)\u001b[0m 2022-03-15 16:58:42,704,704 DEBUG [connectionpool.py:_make_request:456] https://federal.oss-cn-hangzhou.aliyuncs.com:443 \"GET /dataset/public/bank_alice/bank.csv HTTP/1.1\" 200 434679\n",
      "\u001b[2m\u001b[36m(_run pid=73213)\u001b[0m 2022-03-15 16:58:42,674,674 DEBUG [connectionpool.py:_new_conn:1001] Starting new HTTPS connection (1): federal.oss-cn-hangzhou.aliyuncs.com:443\n",
      "\u001b[2m\u001b[36m(_run pid=73213)\u001b[0m 2022-03-15 16:58:42,731,731 DEBUG [connectionpool.py:_make_request:456] https://federal.oss-cn-hangzhou.aliyuncs.com:443 \"GET /dataset/public/bank_bob/bank.csv HTTP/1.1\" 200 596005\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[2m\u001b[36m(PPURuntime pid=73212)\u001b[0m [2022-03-15 16:58:42.898] [info] [context.cc:111] connected to mesh, id=root, self=1\n",
      "\u001b[2m\u001b[36m(PPURuntime pid=73212)\u001b[0m [2022-03-15 16:58:42.951] [info] [executor_base.cc:231] Begin sanity check for input file: .data/1/psi-input.csv\n",
      "\u001b[2m\u001b[36m(PPURuntime pid=73210)\u001b[0m [2022-03-15 16:58:42.898] [info] [context.cc:111] connected to mesh, id=root, self=0\n",
      "\u001b[2m\u001b[36m(PPURuntime pid=73210)\u001b[0m [2022-03-15 16:58:42.935] [info] [executor_base.cc:231] Begin sanity check for input file: .data/0/psi-input.csv\n",
      "\u001b[2m\u001b[36m(PPURuntime pid=73210)\u001b[0m [2022-03-15 16:58:42.942] [info] [executor_base.cc:181] Executing duplicated scripts: LC_ALL=C sort --buffer-size=1G --temporary-directory=./ --stable .data/0/psi-input.csv.keys.1647334722935924684 | LC_ALL=C uniq -d > .data/0/psi-input.csv.duplicated.1647334722935924684\n",
      "\u001b[2m\u001b[36m(PPURuntime pid=73210)\u001b[0m [2022-03-15 16:58:42.947] [info] [executor_base.cc:184] Finished duplicated scripts: LC_ALL=C sort --buffer-size=1G --temporary-directory=./ --stable .data/0/psi-input.csv.keys.1647334722935924684 | LC_ALL=C uniq -d > .data/0/psi-input.csv.duplicated.1647334722935924684, ret=0\n",
      "\u001b[2m\u001b[36m(PPURuntime pid=73210)\u001b[0m [2022-03-15 16:58:42.947] [info] [executor_base.cc:234] End sanity check for input file: .data/0/psi-input.csv, size=11162\n",
      "\u001b[2m\u001b[36m(PPURuntime pid=73212)\u001b[0m [2022-03-15 16:58:42.959] [info] [executor_base.cc:181] Executing duplicated scripts: LC_ALL=C sort --buffer-size=1G --temporary-directory=./ --stable .data/1/psi-input.csv.keys.1647334722951727590 | LC_ALL=C uniq -d > .data/1/psi-input.csv.duplicated.1647334722951727590\n",
      "\u001b[2m\u001b[36m(PPURuntime pid=73212)\u001b[0m [2022-03-15 16:58:42.963] [info] [executor_base.cc:184] Finished duplicated scripts: LC_ALL=C sort --buffer-size=1G --temporary-directory=./ --stable .data/1/psi-input.csv.keys.1647334722951727590 | LC_ALL=C uniq -d > .data/1/psi-input.csv.duplicated.1647334722951727590, ret=0\n",
      "\u001b[2m\u001b[36m(PPURuntime pid=73212)\u001b[0m [2022-03-15 16:58:42.963] [info] [executor_base.cc:234] End sanity check for input file: .data/1/psi-input.csv, size=11162\n",
      "\u001b[2m\u001b[36m(PPURuntime pid=73212)\u001b[0m [2022-03-15 16:58:42.963] [info] [executor_base.cc:249] skip doing psi, because dataset has been aligned!\n",
      "\u001b[2m\u001b[36m(PPURuntime pid=73212)\u001b[0m [2022-03-15 16:58:42.963] [info] [executor_base.cc:255] Begin post filtering, indices.size=11162, should_sort=true\n",
      "\u001b[2m\u001b[36m(PPURuntime pid=73212)\u001b[0m [2022-03-15 16:58:42.967] [info] [executor_base.cc:274] End post filtering, in=.data/1/psi-input.csv, out=.data/1/psi-output.csv\n",
      "\u001b[2m\u001b[36m(PPURuntime pid=73210)\u001b[0m [2022-03-15 16:58:42.963] [info] [executor_base.cc:249] skip doing psi, because dataset has been aligned!\n",
      "\u001b[2m\u001b[36m(PPURuntime pid=73210)\u001b[0m [2022-03-15 16:58:42.963] [info] [executor_base.cc:255] Begin post filtering, indices.size=11162, should_sort=true\n",
      "\u001b[2m\u001b[36m(PPURuntime pid=73210)\u001b[0m [2022-03-15 16:58:42.966] [info] [executor_base.cc:274] End post filtering, in=.data/0/psi-input.csv, out=.data/0/psi-output.csv\n"
     ]
    }
   ],
   "source": [
    "from secretflow.data.vertical import read_csv\n",
    "data_dict = {alice: 'https://federal.oss-cn-hangzhou.aliyuncs.com/dataset/public/bank_alice/bank.csv',\n",
    "               bob: 'https://federal.oss-cn-hangzhou.aliyuncs.com/dataset/public/bank_bob/bank.csv'}\n",
    "\n",
    "vdf = read_csv(data_dict,ppu=ppu,keys='id',drop_keys=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`vdf`为构建好的垂直联邦表，他从全局上只拥有所有数据的`Schema`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['age', 'job', 'marital', 'education', 'y', 'default', 'balance',\n",
       "       'housing', 'loan', 'contact', 'day', 'month', 'duration', 'campaign',\n",
       "       'pdays', 'previous', 'poutcome'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vdf.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们进一步来探索一下vdf的数据管理  \n",
    "通过一个实例可以看出，age这个字段是属于alice的，所以在alice方的partition可以得到对应的列，但是bob方想要去获取age的时候会报`KeyError`错误。  \n",
    "这里有一个Partition的概念，是我们定义的一个数据分片，每个partition都会有自己的device归属，只有归属的device才可以操作数据。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<secretflow.device.device.pyu.PYUObject object at 0x7f602a47c970>\n"
     ]
    },
    {
     "ename": "KeyError",
     "evalue": "<secretflow.device.device.pyu.PYU object at 0x7f5f93bdfbb0>",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "Input \u001b[0;32mIn [11]\u001b[0m, in \u001b[0;36m<cell line: 2>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[38;5;28mprint\u001b[39m(vdf[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mage\u001b[39m\u001b[38;5;124m'\u001b[39m]\u001b[38;5;241m.\u001b[39mpartitions[alice]\u001b[38;5;241m.\u001b[39mdata)\n\u001b[0;32m----> 2\u001b[0m \u001b[38;5;28mprint\u001b[39m(\u001b[43mvdf\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mage\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m]\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mpartitions\u001b[49m\u001b[43m[\u001b[49m\u001b[43mbob\u001b[49m\u001b[43m]\u001b[49m)\n",
      "\u001b[0;31mKeyError\u001b[0m: <secretflow.device.device.pyu.PYU object at 0x7f5f93bdfbb0>"
     ]
    }
   ],
   "source": [
    "print(vdf['age'].partitions[alice].data)\n",
    "print(vdf['age'].partitions[bob])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们接着对生成的联邦表做数据预处理。  \n",
    "我们这里以LabelEncoder和MinMaxScaler为例，这两个预处理函数在`sklearn`中有对应的概念，他的使用方法和sklearn中也是类似的"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "from secretflow.preprocessing.scaler import MinMaxScaler\n",
    "from secretflow.preprocessing.encoder import LabelEncoder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "encoder = LabelEncoder()\n",
    "vdf['job'] = encoder.fit_transform(vdf['job'])\n",
    "vdf['marital'] = encoder.fit_transform(vdf['marital'])\n",
    "vdf['education'] = encoder.fit_transform(vdf['education'])\n",
    "vdf['default'] = encoder.fit_transform(vdf['default'])\n",
    "vdf['housing'] = encoder.fit_transform(vdf['housing'])\n",
    "vdf['loan'] = encoder.fit_transform(vdf['loan'])\n",
    "vdf['contact'] = encoder.fit_transform(vdf['contact'])\n",
    "vdf['poutcome'] = encoder.fit_transform(vdf['poutcome'])\n",
    "vdf['month'] = encoder.fit_transform(vdf['month'])\n",
    "vdf['y'] = encoder.fit_transform(vdf['y'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们将数据拆分成data和label两部分"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "label = vdf['y']\n",
    "data = vdf.drop(columns='y', inplace=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "label= <class 'secretflow.data.vertical.dataframe.VDataFrame'>,\n",
      "data = <class 'secretflow.data.vertical.dataframe.VDataFrame'>\n"
     ]
    }
   ],
   "source": [
    "print(f\"label= {type(label)},\\ndata = {type(data)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通过MinMaxScaler做数据标准化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[2m\u001b[36m(_run pid=73204)\u001b[0m /home/xingmeng.zhxm/anaconda3/envs/secretflow/lib/python3.8/site-packages/sklearn/base.py:443: UserWarning: X has feature names, but MinMaxScaler was fitted without feature names\n",
      "\u001b[2m\u001b[36m(_run pid=73204)\u001b[0m   warnings.warn(\n",
      "\u001b[2m\u001b[36m(_run pid=73213)\u001b[0m /home/xingmeng.zhxm/anaconda3/envs/secretflow/lib/python3.8/site-packages/sklearn/base.py:443: UserWarning: X has feature names, but MinMaxScaler was fitted without feature names\n",
      "\u001b[2m\u001b[36m(_run pid=73213)\u001b[0m   warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "scaler = MinMaxScaler()\n",
    "\n",
    "data = scaler.fit_transform(vdf[list(data.columns)])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接着我们将数据集划分成train-set和test-set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "from secretflow.data.split import train_test_split\n",
    "random_state = 1234\n",
    "train_data,test_data = train_test_split(data,train_size=0.8,random_state=random_state)\n",
    "train_label,test_label = train_test_split(label,train_size=0.8,random_state=random_state)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**小结：**到这里为止，我们就完成了**联邦表的定义**，**数据预处理**，以及**训练集和测试集的划分**  \n",
    "secretflow框架定义了跨越多方的`联邦表`概念，同时定义了一套构建在联邦表上的操作，逻辑对等`pandas.DataFrame`，同时定义了对于联邦表的预处理操作，逻辑对等`sklearn`,您在使用过程中遇到问题，可以参考我们的文档以及api介绍，进一步了解其他的功能"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型介绍"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**单机版本**：  \n",
    "对于该任务一个基本的DNN就可以完成，输入16维特征，经过一个DNN网络，输出对于正负样本的概率。\n",
    "\n",
    "**联邦版本**：\n",
    "* Alice：\n",
    "    - base_net:输入4维特征，经过一个dnn网络得到hidden\n",
    "    - fuse_net:接收自己的hidden_alice,以及bob计算得到的hidden特征，输入的fuse_net，进行特征融合，送入之后的网络完成整个forward过程和backward过程\n",
    "* Bob：\n",
    "    - base_net:输入12维特征，经过一个dnn网络得到hidden，然后将hidden发送给alice方，完成接下来的运算"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 定义模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来我们开始创建联邦模型  \n",
    "在垂直场景我们定义了`SLTFModel`和`SLTorchModel(WIP)`,用于构建垂直场景的拆分学习，我们定义了简单易用可扩展的接口，可以很方便的将您已有的模型，转换成SF—Model，进而进行垂直场景联邦建模。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "拆分学习即将一个模型拆分开来，一部分放在数据的本地执行，另外一部分放在有label的一方，或者server端执行。  \n",
    "首先我们来定义本地执行的模型——base_model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 创建base模型\n",
    "def create_base_model(input_dim, output_dim,  name='base_model'):\n",
    "    # Create model\n",
    "    def create_model():\n",
    "        from tensorflow import keras\n",
    "        from tensorflow.keras import layers\n",
    "        import tensorflow as tf\n",
    "        model = keras.Sequential(\n",
    "            [\n",
    "                keras.Input(shape=input_dim),\n",
    "                layers.Dense(100,activation =\"relu\" ),\n",
    "                layers.Dense(output_dim, activation=\"relu\"),\n",
    "            ]\n",
    "        )\n",
    "        # Compile model\n",
    "        model.summary()\n",
    "        model.compile(loss='binary_crossentropy',\n",
    "                      optimizer='adam',\n",
    "                      metrics=[\"accuracy\",tf.keras.metrics.AUC()])\n",
    "        return model  # 不能序列化的\n",
    "    return create_model\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们使用create_base_model分别为`alice`和`bob`创建他们的base model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# prepare model\n",
    "hidden_size = 64\n",
    "# 用户定义的已编译后的keras model\n",
    "model_base_alice = create_base_model(4, hidden_size)\n",
    "model_base_bob = create_base_model(12, hidden_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " dense (Dense)               (None, 100)               500       \n",
      "                                                                 \n",
      " dense_1 (Dense)             (None, 64)                6464      \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 6,964\n",
      "Trainable params: 6,964\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Model: \"sequential_1\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " dense_2 (Dense)             (None, 100)               1300      \n",
      "                                                                 \n",
      " dense_3 (Dense)             (None, 64)                6464      \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 7,764\n",
      "Trainable params: 7,764\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-03-15 17:00:02.531874: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib:/opt/rh/devtoolset-10/root/usr/lib64/dyninst:/opt/rh/devtoolset-10/root/usr/lib/dyninst:/opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib\n",
      "2022-03-15 17:00:02.531908: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.engine.sequential.Sequential at 0x7f60299d43d0>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_base_alice()\n",
    "model_base_bob()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来我们定义有label的一方，或者server端的模型——fuse_model  \n",
    "在fuse_model的定义中，我们需要正确的定义loss，optimizer，metrics。这里可以兼容所有您已有的keras模型的配置"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "#创建fuse模型\n",
    "def create_fuse_model(input_dim, output_dim, party_nums, name='fuse_model'):\n",
    "    def create_model():\n",
    "        from tensorflow import keras\n",
    "        from tensorflow.keras import layers\n",
    "        import tensorflow as tf\n",
    "        # input\n",
    "        input_layers = []\n",
    "        for i in range(party_nums):\n",
    "            input_layers.append(keras.Input(input_dim,))\n",
    "        \n",
    "        # 定义融合逻辑\n",
    "        merged_layer = layers.concatenate(input_layers)\n",
    "        fuse_layer = layers.Dense(64, activation='relu')(merged_layer)\n",
    "        output = layers.Dense(output_dim, activation='sigmoid')(fuse_layer)\n",
    "        # 构建模型\n",
    "        model = keras.Model(inputs=input_layers, outputs=output)\n",
    "        model.summary()\n",
    "        # 编译模型，定义损失，优化器，以及指标\n",
    "        model.compile(loss='binary_crossentropy',\n",
    "                      optimizer='adam',\n",
    "                      metrics=[\"accuracy\",tf.keras.metrics.AUC()])\n",
    "        return model\n",
    "    return create_model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 定义融合模型\n",
    "model_fuse = create_fuse_model(\n",
    "    input_dim=hidden_size, party_nums=2, output_dim=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model\"\n",
      "__________________________________________________________________________________________________\n",
      " Layer (type)                   Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      " input_3 (InputLayer)           [(None, 64)]         0           []                               \n",
      "                                                                                                  \n",
      " input_4 (InputLayer)           [(None, 64)]         0           []                               \n",
      "                                                                                                  \n",
      " concatenate (Concatenate)      (None, 128)          0           ['input_3[0][0]',                \n",
      "                                                                  'input_4[0][0]']                \n",
      "                                                                                                  \n",
      " dense_4 (Dense)                (None, 64)           8256        ['concatenate[0][0]']            \n",
      "                                                                                                  \n",
      " dense_5 (Dense)                (None, 1)            65          ['dense_4[0][0]']                \n",
      "                                                                                                  \n",
      "==================================================================================================\n",
      "Total params: 8,321\n",
      "Trainable params: 8,321\n",
      "Non-trainable params: 0\n",
      "__________________________________________________________________________________________________\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.engine.functional.Functional at 0x7f602a34d8b0>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_fuse()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 创建拆分学习模型\n",
    "secretflow提供了拆分学习的模型 SLModelTF  \n",
    "SLModelTF模型初始化需要3个参数\n",
    "* base_model_dict：一个字典需要传入参与训练的所有client以及base_model映射\n",
    "* device_y：PYU，哪一方持有label\n",
    "* model_fuse：融合模型，具体的优化器以及损失函数都在这个模型中进行定义"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "定义base_model_dict  \n",
    "```python\n",
    "base_model_dict:Dict[PYU,model_fn]\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "base_model_dict = {\n",
    "    alice: model_base_alice,\n",
    "    bob:   model_base_bob\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[2m\u001b[36m(_run pid=73204)\u001b[0m 2022-03-15 17:00:14.728845: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib:/opt/rh/devtoolset-10/root/usr/lib64/dyninst:/opt/rh/devtoolset-10/root/usr/lib/dyninst:/opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib\n",
      "\u001b[2m\u001b[36m(_run pid=73213)\u001b[0m 2022-03-15 17:00:14.728845: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib:/opt/rh/devtoolset-10/root/usr/lib64/dyninst:/opt/rh/devtoolset-10/root/usr/lib/dyninst:/opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib\n",
      "\u001b[2m\u001b[36m(_run pid=73204)\u001b[0m 2022-03-15 17:00:15,383,383 DEBUG [tpu_cluster_resolver.py:<module>:35] Falling back to TensorFlow client; we recommended you install the Cloud TPU client directly with pip install cloud-tpu-client.\n",
      "\u001b[2m\u001b[36m(_run pid=73213)\u001b[0m 2022-03-15 17:00:15,385,385 DEBUG [tpu_cluster_resolver.py:<module>:35] Falling back to TensorFlow client; we recommended you install the Cloud TPU client directly with pip install cloud-tpu-client.\n",
      "\u001b[2m\u001b[36m(_run pid=73204)\u001b[0m 2022-03-15 17:00:15,568,568 DEBUG [__init__.py:<module>:47] Creating converter from 7 to 5\n",
      "\u001b[2m\u001b[36m(_run pid=73204)\u001b[0m 2022-03-15 17:00:15,568,568 DEBUG [__init__.py:<module>:47] Creating converter from 5 to 7\n",
      "\u001b[2m\u001b[36m(_run pid=73204)\u001b[0m 2022-03-15 17:00:15,568,568 DEBUG [__init__.py:<module>:47] Creating converter from 7 to 5\n",
      "\u001b[2m\u001b[36m(_run pid=73204)\u001b[0m 2022-03-15 17:00:15,568,568 DEBUG [__init__.py:<module>:47] Creating converter from 5 to 7\n",
      "\u001b[2m\u001b[36m(_run pid=73213)\u001b[0m 2022-03-15 17:00:15,571,571 DEBUG [__init__.py:<module>:47] Creating converter from 7 to 5\n",
      "\u001b[2m\u001b[36m(_run pid=73213)\u001b[0m 2022-03-15 17:00:15,571,571 DEBUG [__init__.py:<module>:47] Creating converter from 5 to 7\n",
      "\u001b[2m\u001b[36m(_run pid=73213)\u001b[0m 2022-03-15 17:00:15,571,571 DEBUG [__init__.py:<module>:47] Creating converter from 7 to 5\n",
      "\u001b[2m\u001b[36m(_run pid=73213)\u001b[0m 2022-03-15 17:00:15,571,571 DEBUG [__init__.py:<module>:47] Creating converter from 5 to 7\n"
     ]
    }
   ],
   "source": [
    "sl_model = SLModelTF(\n",
    "    base_model_dict=base_model_dict, \n",
    "    device_y=alice,  \n",
    "    model_fuse=model_fuse)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m 2022-03-15 17:00:16.946573: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib:/opt/rh/devtoolset-10/root/usr/lib64/dyninst:/opt/rh/devtoolset-10/root/usr/lib/dyninst:/opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m 2022-03-15 17:00:16.946600: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m 2022-03-15 17:00:16.945592: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib:/opt/rh/devtoolset-10/root/usr/lib64/dyninst:/opt/rh/devtoolset-10/root/usr/lib/dyninst:/opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m 2022-03-15 17:00:16.945622: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)\n",
      "2022-03-15 17:00:17.081 | DEBUG    | secretflow.model.sl_model:fit:117 - validation_data provided\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m Model: \"sequential\"\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m _________________________________________________________________\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m  Layer (type)                Output Shape              Param #   \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m =================================================================\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m  dense (Dense)               (None, 100)               1300      \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m                                                                  \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m  dense_1 (Dense)             (None, 64)                6464      \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m                                                                  \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m =================================================================\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m Total params: 7,764\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m Trainable params: 7,764\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m Non-trainable params: 0\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m _________________________________________________________________\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m Model: \"sequential_1\"\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m _________________________________________________________________\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m  Layer (type)                Output Shape              Param #   \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m =================================================================\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m  dense_2 (Dense)             (None, 100)               1300      \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m                                                                  \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m  dense_3 (Dense)             (None, 64)                6464      \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m                                                                  \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m =================================================================\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m Total params: 7,764\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m Trainable params: 7,764\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m Non-trainable params: 0\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73204)\u001b[0m _________________________________________________________________\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m Model: \"sequential\"\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m _________________________________________________________________\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m  Layer (type)                Output Shape              Param #   \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m =================================================================\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m  dense (Dense)               (None, 100)               500       \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m                                                                  \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m  dense_1 (Dense)             (None, 64)                6464      \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m                                                                  \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m =================================================================\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m Total params: 6,964\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m Trainable params: 6,964\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m Non-trainable params: 0\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m _________________________________________________________________\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m Model: \"sequential_1\"\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m _________________________________________________________________\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m  Layer (type)                Output Shape              Param #   \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m =================================================================\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m  dense_2 (Dense)             (None, 100)               500       \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m                                                                  \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m  dense_3 (Dense)             (None, 64)                6464      \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m                                                                  \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m =================================================================\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m Total params: 6,964\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m Trainable params: 6,964\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m Non-trainable params: 0\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m _________________________________________________________________\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m Model: \"model\"\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m __________________________________________________________________________________________________\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m  Layer (type)                   Output Shape         Param #     Connected to                     \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m ==================================================================================================\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m  input_3 (InputLayer)           [(None, 64)]         0           []                               \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m                                                                                                   \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m  input_4 (InputLayer)           [(None, 64)]         0           []                               \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m                                                                                                   \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m  concatenate (Concatenate)      (None, 128)          0           ['input_3[0][0]',                \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m                                                                   'input_4[0][0]']                \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m                                                                                                   \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m  dense_4 (Dense)                (None, 64)           8256        ['concatenate[0][0]']            \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m                                                                                                   \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m  dense_5 (Dense)                (None, 1)            65          ['dense_4[0][0]']                \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m                                                                                                   \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m ==================================================================================================\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m Total params: 8,321\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m Trainable params: 8,321\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m Non-trainable params: 0\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=73213)\u001b[0m __________________________________________________________________________________________________\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-03-15 17:00:19.430 | INFO     | secretflow.model.sl_model:fit:159 - valid evaluate={'loss': 0.61008704, 'accuracy': 0.7113402, 'auc_2': 0.73550725}\n",
      "2022-03-15 17:00:21.394 | INFO     | secretflow.model.sl_model:fit:159 - valid evaluate={'loss': 0.55679965, 'accuracy': 0.7010309, 'auc_2': 0.8009378}\n",
      "2022-03-15 17:00:23.390 | INFO     | secretflow.model.sl_model:fit:159 - valid evaluate={'loss': 0.5069495, 'accuracy': 0.7525773, 'auc_2': 0.8307758}\n",
      "2022-03-15 17:00:25.317 | INFO     | secretflow.model.sl_model:fit:159 - valid evaluate={'loss': 0.49515337, 'accuracy': 0.78350514, 'auc_2': 0.83567774}\n",
      "2022-03-15 17:00:27.265 | INFO     | secretflow.model.sl_model:fit:159 - valid evaluate={'loss': 0.48970804, 'accuracy': 0.78350514, 'auc_2': 0.83823526}\n",
      "2022-03-15 17:00:29.204 | INFO     | secretflow.model.sl_model:fit:159 - valid evaluate={'loss': 0.461964, 'accuracy': 0.814433, 'auc_2': 0.8572037}\n",
      "2022-03-15 17:00:31.091 | INFO     | secretflow.model.sl_model:fit:159 - valid evaluate={'loss': 0.5079753, 'accuracy': 0.77319586, 'auc_2': 0.83034956}\n",
      "2022-03-15 17:00:33.023 | INFO     | secretflow.model.sl_model:fit:159 - valid evaluate={'loss': 0.50267136, 'accuracy': 0.77319586, 'auc_2': 0.82757884}\n",
      "2022-03-15 17:00:34.951 | INFO     | secretflow.model.sl_model:fit:159 - valid evaluate={'loss': 0.52231234, 'accuracy': 0.742268, 'auc_2': 0.83972716}\n",
      "2022-03-15 17:00:36.866 | INFO     | secretflow.model.sl_model:fit:159 - valid evaluate={'loss': 0.4776467, 'accuracy': 0.7938144, 'auc_2': 0.85507244}\n"
     ]
    }
   ],
   "source": [
    "sl_model.fit(train_data, train_label,validation_data=(test_data,test_label), epochs=10, batch_size=128, shuffle=True,verbose=1,validation_freq=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们来调用一下评估函数，看下训练效果怎么样"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 0.4776467, 'accuracy': 0.7938144, 'auc_2': 0.85507244}\n"
     ]
    }
   ],
   "source": [
    "global_metric = sl_model.evaluate(test_data, test_label, batch_size=128)\n",
    "print(global_metric)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 和单方模型的对比"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 模型\n",
    "模型结构和上面split learning的模型保持一致，但是这里只用了有label的alice方的模型结构，模型定义参考下面的代码\n",
    "#### 数据\n",
    "数据同样使用kaggle的反欺诈数据，单方模型这里我们只是用了新银行alice方数据\n",
    "1. 样本量总计11162个，其中训练集8929， 测试集2233\n",
    "2. 特征4维"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow import keras\n",
    "from tensorflow.keras import layers\n",
    "import tensorflow as tf\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "def create_model():\n",
    "\n",
    "    model = keras.Sequential(\n",
    "        [\n",
    "            keras.Input(shape=4),\n",
    "            layers.Dense(100,activation =\"relu\" ),\n",
    "            layers.Dense(64, activation='relu'),\n",
    "            layers.Dense(64, activation='relu'),\n",
    "            layers.Dense(1, activation='sigmoid')\n",
    "        ]\n",
    "    )\n",
    "    model.compile(loss='binary_crossentropy',\n",
    "                      optimizer='adam',\n",
    "                      metrics=[\"accuracy\",tf.keras.metrics.AUC()])\n",
    "    return model\n",
    "\n",
    "single_model = create_model()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数据预处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset_dict['alice']= dataset_dict['alice'].drop(columns=\"id\",inplace=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>job</th>\n",
       "      <th>marital</th>\n",
       "      <th>education</th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>59</td>\n",
       "      <td>admin.</td>\n",
       "      <td>married</td>\n",
       "      <td>secondary</td>\n",
       "      <td>yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>56</td>\n",
       "      <td>admin.</td>\n",
       "      <td>married</td>\n",
       "      <td>secondary</td>\n",
       "      <td>yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>41</td>\n",
       "      <td>technician</td>\n",
       "      <td>married</td>\n",
       "      <td>secondary</td>\n",
       "      <td>yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>55</td>\n",
       "      <td>services</td>\n",
       "      <td>married</td>\n",
       "      <td>secondary</td>\n",
       "      <td>yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>54</td>\n",
       "      <td>admin.</td>\n",
       "      <td>married</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11157</th>\n",
       "      <td>33</td>\n",
       "      <td>blue-collar</td>\n",
       "      <td>single</td>\n",
       "      <td>primary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11158</th>\n",
       "      <td>39</td>\n",
       "      <td>services</td>\n",
       "      <td>married</td>\n",
       "      <td>secondary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11159</th>\n",
       "      <td>32</td>\n",
       "      <td>technician</td>\n",
       "      <td>single</td>\n",
       "      <td>secondary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11160</th>\n",
       "      <td>43</td>\n",
       "      <td>technician</td>\n",
       "      <td>married</td>\n",
       "      <td>secondary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11161</th>\n",
       "      <td>34</td>\n",
       "      <td>technician</td>\n",
       "      <td>married</td>\n",
       "      <td>secondary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>11162 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       age          job  marital  education    y\n",
       "0       59       admin.  married  secondary  yes\n",
       "1       56       admin.  married  secondary  yes\n",
       "2       41   technician  married  secondary  yes\n",
       "3       55     services  married  secondary  yes\n",
       "4       54       admin.  married   tertiary  yes\n",
       "...    ...          ...      ...        ...  ...\n",
       "11157   33  blue-collar   single    primary   no\n",
       "11158   39     services  married  secondary   no\n",
       "11159   32   technician   single  secondary   no\n",
       "11160   43   technician  married  secondary   no\n",
       "11161   34   technician  married  secondary   no\n",
       "\n",
       "[11162 rows x 5 columns]"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset_dict['alice']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "alice_data = dataset_dict['alice']\n",
    "encoder = LabelEncoder()\n",
    "alice_data['job'] = encoder.fit_transform(alice_data['job'])\n",
    "alice_data['marital'] = encoder.fit_transform(alice_data['marital'])\n",
    "alice_data['education'] = encoder.fit_transform(alice_data['education'])\n",
    "alice_data['y'] =  encoder.fit_transform(alice_data['y'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = alice_data['y']\n",
    "alice_data = alice_data.drop(columns=['y'],inplace=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "scaler = MinMaxScaler()\n",
    "alice_data = scaler.fit_transform(alice_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.53246753, 0.        , 0.5       , 0.33333333],\n",
       "       [0.49350649, 0.        , 0.5       , 0.33333333],\n",
       "       [0.2987013 , 0.81818182, 0.5       , 0.33333333],\n",
       "       ...,\n",
       "       [0.18181818, 0.81818182, 1.        , 0.33333333],\n",
       "       [0.32467532, 0.81818182, 0.5       , 0.33333333],\n",
       "       [0.20779221, 0.81818182, 0.5       , 0.33333333]])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "alice_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_data,test_data = train_test_split(alice_data,train_size=0.8,random_state=random_state)\n",
    "train_label,test_label = train_test_split(y,train_size=0.8,random_state=random_state)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "70/70 [==============================] - 1s 7ms/step - loss: 0.6875 - accuracy: 0.5544 - auc_3: 0.5567 - val_loss: 0.6819 - val_accuracy: 0.5705 - val_auc_3: 0.6014\n",
      "Epoch 2/10\n",
      "70/70 [==============================] - 0s 2ms/step - loss: 0.6769 - accuracy: 0.5821 - auc_3: 0.6010 - val_loss: 0.6749 - val_accuracy: 0.5723 - val_auc_3: 0.6068\n",
      "Epoch 3/10\n",
      "70/70 [==============================] - 0s 2ms/step - loss: 0.6705 - accuracy: 0.5913 - auc_3: 0.6120 - val_loss: 0.6723 - val_accuracy: 0.5755 - val_auc_3: 0.6115\n",
      "Epoch 4/10\n",
      "70/70 [==============================] - 0s 2ms/step - loss: 0.6673 - accuracy: 0.5940 - auc_3: 0.6181 - val_loss: 0.6712 - val_accuracy: 0.5768 - val_auc_3: 0.6134\n",
      "Epoch 5/10\n",
      "70/70 [==============================] - 0s 2ms/step - loss: 0.6652 - accuracy: 0.5950 - auc_3: 0.6227 - val_loss: 0.6703 - val_accuracy: 0.5790 - val_auc_3: 0.6164\n",
      "Epoch 6/10\n",
      "70/70 [==============================] - 0s 2ms/step - loss: 0.6638 - accuracy: 0.5979 - auc_3: 0.6254 - val_loss: 0.6698 - val_accuracy: 0.5826 - val_auc_3: 0.6177\n",
      "Epoch 7/10\n",
      "70/70 [==============================] - 0s 2ms/step - loss: 0.6626 - accuracy: 0.5982 - auc_3: 0.6282 - val_loss: 0.6692 - val_accuracy: 0.5822 - val_auc_3: 0.6195\n",
      "Epoch 8/10\n",
      "70/70 [==============================] - 0s 2ms/step - loss: 0.6618 - accuracy: 0.6014 - auc_3: 0.6299 - val_loss: 0.6691 - val_accuracy: 0.5835 - val_auc_3: 0.6200\n",
      "Epoch 9/10\n",
      "70/70 [==============================] - 0s 2ms/step - loss: 0.6610 - accuracy: 0.6014 - auc_3: 0.6314 - val_loss: 0.6688 - val_accuracy: 0.5808 - val_auc_3: 0.6207\n",
      "Epoch 10/10\n",
      "70/70 [==============================] - 0s 2ms/step - loss: 0.6605 - accuracy: 0.6020 - auc_3: 0.6324 - val_loss: 0.6687 - val_accuracy: 0.5831 - val_auc_3: 0.6209\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f6014fb9d60>"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "single_model.fit(train_data,train_label,validation_data=(test_data,test_label),batch_size=128,epochs=10,shuffle=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 小结\n",
    "上面两个实验模拟了一个典型的垂直场景的训练问题，alice和bob拥有相同的样本群，但每一方只有样本的一部分数据，如果alice只用自己的一方数据来训练模型，能够得到一个精确度0.583,auc 0.62的模型，但是如果联合bob的数据之后，可以获得一个精确度0.793，auc0.855的模型。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 总结"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 本篇我们介绍了什么是拆分学习，以及如何在secretflow框架下进行拆分学习  \n",
    "* 从实验数据可以看出，split learning在扩充样本维度，通过联合多方训练提升模型效果方面有显著优势\n",
    "* 本文档使用明文聚合来做演示，同时没有考虑隐层的泄露问题，secretflow提供了AggLayer通过MPC,TEE,HE，以及DP等方式规避隐层明文传输泄露的问题，感兴趣可以看相关文档。\n",
    "* 下一步，你可能想尝试不同的数据集，您需要先将数据集进行垂直切分，然后按照本教程的流程进行\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
